This invention is a learning binary network system characterized, by consisting of an input layer, a coupling layer, a first binary gate layer with first similar logical elements, a second binary gate layer with second similar logical elements, and an output layer, in that each coupling condition between the adjacent layers limited to one way directing from their inlet side to the outlet side, and each layer has independent routes without mutual coupling conditions, the coupling layer having means for selecting either one of a direct coupling condition and a coupling condition routed through an inverter, relative to routes from the respective signal units in the input layer to the respective signal units in the first binary gate layer in the learning network.
1. The Technical Field of this Invention
This invention relates to a binary learning system applied to such as character recognition, robot motion control and association memory.
2. The Prior Arts
A learning system has been known as a neural network. The neural network means a circuit consisting of suspected elements of neural cells (neuron) in a network in the same as basic information treating units for function of neural cells of a living body, the neural network applied to such as character recognition, robot motion control and association memory.
A neuron NE as one unit indicated in FIG. 17, consists of means for receiving input signal from the other neuron, means for changing the input signal under a determined rule in coupling conditions, means for limiting its changing threshold, and means for putting out the changed output. And in the coupling condition to the other neuron, a weight [Wij] indicating a coupling power is additionally attached to the unit.
The coupling condition of this neuron includes an exciting coupling (Wij greater than 0)indicating a condition in that the more increase input of its self is then the more increase input from the other neuron, and a suppressing coupling (Wij less than 0) indicating a condition in that the more decrease input of its self is then the more increase input from the other neuron in reverse. Then the changing of this weight [Wij] and threshold xcex8i course the changed constitution of the network.
FIG. 18 indicates neutral network consisting of neuron NE mentioned above, which includes an input layer, a medium layer and an output layer, each layer having no coupling condition therein, and the medium layer being capable of use of a plurality of layers. Such network is actuated so as to propagate the input signal of the input layer to the medium layer, and then the signal therein being changed with coupling coefficient, or weight and threshold, as resultant of propagating to the output layer. In the output layer, the signal is further treated as an output signal Z by addition of any weight and threshold.
Input NEX in the medium and output layers is counted by FORMULA 19 mentioned hereinafter.
Then, the neuron input NEX puts out after non-linear treatment. Further, output Yj or Zj in the medium and output layers is obtained by Sigmoid coefficient so indicated as to FORMULA 20 in general.
In this case, the leaning means to change the weight and threshold to decrease or put preferably to zero, an error between the real output Zj and a prescribed output Tj (monitor signal). This changing value is given in a manner to use error-propagating in reverse, and the changing value in the formula mentioned above is different to the neuron in the input layer or medium layer.
In analog circuits applied to the mentioned network, signal intensity of the input or output exists as a voltage, the weight of the neuron is a resistance existing on each neuron line, and the neuron output coefficient (Sigmoid coefficient) is a propagating coefficient of an amplifier. And in order to indicate the exiting and suppressing couples between the neurons, the output of the amplifier is separated to two outputs, so as to generate a plus or minus signal by reversing one of the outputs through an inverter.
The mentioned system including the analog circuits involves problems mentioned hereinafter.
a. Irregular operations of neuron elements are generated due to the temperature property.
b. As to control the error between the real output Z and monitor output T mentioned above, the error revisable circuit is complicated as well as the error introducing to zero is difficult.
c. The use of amplifier induces the further complication and large size of circuit, and the operating time is extended and causes in such difficulty that the corporation network is not produced.
Already, digital circuits for neutral network are proposed in the Japanese publication document of the patent application, its publication number being 108594/93. In this case, all neuron elements consists of logic elements without the irregular operations due to their temperature property.
However, the system having the digital circuits mentioned above involves problems described hereinafter.
a. For signal propagation between the respective layers, pulse uses so as to indicate pulse density (pulse counts per unit time) as amount of analog signal. Accordingly, this system is incapable of error control into zero and wastes long operation time.
b. The volume of each neuron element is increased, thereby causing the largest and expanded construction of the neutral network.
c. As learning, the changing value of the weight and threshold must be controlled in the respective medium and output layers under use of the prior constitution of the neutral network.
In order to solve the problems mentioned above, this inventor had proposed a new binary learning system consisting of logical elements as shown in FIG. 1, disclosed to U.S patent application Ser. No. 744,299/96.
The learning network consists of an input layer 21 having a plurality of binary input terminals X1, X2, . . . Xn, a coupling layer 22, a first binary gate layer (AND layer) 23 with a plurality of similar logical elements (for instance, AND logic), a second binary gate layer (OR layer) 24 with a plurality of similar logical elements (for instance, OR logic), and an output layer 25, the respective layers having no coupled therein and the coupling condition between the mutual layers being limited to a way only directed from the input layer to the output layer (Feed forward type).
The couple in the coupling layer, between each units of the input layer and each units of AND layer, is selected to coupling conditions mentioned hereinafter.
(1) direct coupling
(2) coupling through an inverter
(3) all [1] coupling
(4) all [0] coupling
The coupling layer applied to the coupling conditions mentioned above can consist of suspected neurons, and then the learning network is produced as shown in FIG. 14.
In this case, one unit of OR layer 24 is only shown in FIG. 14 for easy explanation, and the respective output terminal Z is only one.
The suspected neurons NE, as shown in FIG. 12, exist one input and one output, the weight Wij from the input is either one of 1 or xe2x88x921 and the threshold xcex8ij is selected to xe2x88x921.5, xe2x88x920.5, 0.5 and 1.5.
Then, the output Yij given by the input Xi, weight Wij and threshold xcex8ij is all indicated in four coupling conditions mentioned above. And the output Yij is calculated by FORMULA 3 or FORMULA 4 mentioned hereinafter
or
As learning, error E between the real output Z and monitor output T can be obtained by FORMULA 5 as nextly mentioned.
In this case, the learning is accomplished with control of the weight Wij and threshold xcex8ij, as similar to the prior idea. When the weight Wij and threshold xcex8ij are controlled according to error E lowering downwards in the highest speed, their control values xcex94W and xcex94xcex8are obtained by FORMULA 1 or FORMULA 6.
or
xcex5w, xcex5xcex8 are to plus and are calculated as mentioned hereinafter, by the learning rule under use of the error propagation in reverse.
The output in this case, is only one, therefore
Accordingly,
Since it relates Z=OR,
As results, the signal at OR gate is resembled by a continuous coefficient mentioned hereinafter.
Herein, M is the maximum value in approximate inputs without ANDi, namely, M=Max(ANDi, i=1, 2, 3 . . . , ixe2x89xa0j), FIG. 15 indicates this real value. Then,
Similarly, the signal at AND gate is resembled by a continuous coefficient mentioned hereinafter.
Herein, m is the minimum value in approximate inputs without ANDi, namely, m=Min(ANDi, i=1, 2, 3 . . . , ixe2x89xa0j), FIG. 16 indicates this real value. Then,
Finally,
Then,
Since it is fxe2x80x2(x) greater than 0, the control values xcex94W of the weight Wij and xcex94xcex8 of the threshold are obtained under fxe2x80x2(x)=1, by FORMULA 17.
If xcex5w=2, xcex5w, xcex5xcex8=1, then
In the mentioned formula, all values are binary count, then the control values indicate the output signal Z, monitor signal T, and the AND output signals ANDi, Yi and Xi as logical forms.
As mentioned above, this case indicates the binary learning system in that NE includes one input and one output, and Wij, xcex8j, Yij, xcex94Wij, xcex94dxcex8ij, etc., are binary indicated, as well as the output Yij of NE is indicated in four coupling conditions mentioned above, thus as the learning operation causes to control the coupling condition between the respective inputs Xi of the input layer and the respective units (AND) of the first gate layer. Accordingly, it is accomplished that the learning network is in a simple constitution, the learning time is rather shortened, and more particularly, the error E is easily induced into zero.
However, the learning network mentioned above and disclosed in the U.S. Patent application, includes four coupling conditions, though the outputs of units of the coupling layer are in binary counts.
If a learning network operated in two coupling conditions may be produced in the present time, it causes the most simple constitution of the units of the coupling layer, the error revising circuit, and preferably, all the learning network in hardware technique.
Accordingly, the learning network mentioned above should be further improved for simple constitution of the network.
This invention is proposed to improve a binary learning network system in that the learning is exactly and quickly accomplished, and the constitution is the most simplified.
For solving various problems mentioned above, this invention newly provides a binary learning system characterized by consisting of an input layer having binary input terminals, a coupling layer, a first binary gate layer with first similar logical elements, a second binary gate layer with second similar logical elements, and an output layer, so as to form a learning network, in that each coupling condition between the adjacent layers limited to one way directing from their inlet side to the outlet side, and each layer has independent routes without mutual coupling conditions, the coupling layer having means for selecting either one of a direct coupling condition and a coupling condition routed through an inverter, relative to routes from the respective signal units in the input layer to the respective signal units in the first binary gate layer, in such manner that the selected coupling condition is adapted to eliminate or decrease the respective errors between original output signals at the output layer and monitor signals in the learning network.
In this invention, the binary learning system mentioned above including a process mentioned herein after:
(1) One of the coupling conditions is so selected as to learn under the case in that the original output signal is different from the monitor signal, and neglect the learning under the case in that the both signals mentioned above are as the same.
(2) The learning is so practiced as to select one of the coupling conditions between the signal units in the input layer and the signal units in the first binary gate layer in order of the unit selection from the highest position to the lowest position in the first binary gate layer, and to select all input terminals in each unit in the same time or the highest position to the lowest position in the input layer.
(3) The learning after selecting the coupling condition to the lowest positioned unit, is again carried on to the highest position as necessary.
In this invention including the binary learning system mentioned above, the first and second logical elements include pluralities of OR gate and AND gate circuits respectively in their orders.
And in the binary learning system mentioned above, the first and second logical elements include pluralities of AND gate and OR gate circuits respectively in their orders.
Further, in the binary learning system mentioned above, the first and second logical elements include pluralities of NAND gate and NAND gate circuits respectively.
In another case of the binary learning system mentioned, the first and second logical elements include pluralities of NOR gate and NOR gate circuits respectively.
Finally, in the binary learning system, the first and second logical elements include pluralities of EXOR gate and EXOR gate circuits respectively.